+1 to the proposal.

Additionally, I'd like to propose a (permissive) convention for directory/files organization.

• Factories should consume directories (vs files)
• All yaml files should contain a dictionary
• Files in a directory should be parsed together and flattened into a single dictionary

This proposal goes against e.g. our current subnet factory where YaML path and filename are used to extract data, and would give developers freedom to implement multiple resources in a single file, or to use one file per resource, as they see fit.

• Factories should consume directories (vs files)

Yes, agreed with exceptions (the permissive bit you mentioned?), e.g. the cidr file in firewall rules.

• All yaml files should contain a dictionary

I'm trying to think where we would need a list and failing, so +1.

• Files in a directory should be parsed together and flattened into a single dictionary

Yep, again with exceptions like the cidr file.

This proposal goes against e.g. our current subnet factory where YaML path and filename are used to extract data, and would give developers freedom to implement multiple resources in a single file, or to use one file per resource, as they see fit.

It's a pattern I kind of like as it allows logical partitioning, and the same for firewall rules where logical partitioning might be useful. But then partitioning can happen at the file name level, and having a (loose) standard is better than having none, so +1.

PS - firewall rules are split into ingress/egress blocks that mimick the variable, and that I would keep so n rules for 1 file
PPS - I would clearly state that factories should mimick and implement the variable interface for the module, including optionals and validation (done in code and checks).

Can you add those bits to the doc by editing this branch?

• Factories should consume directories (vs files)

Yes, agreed with exceptions (the permissive bit you mentioned?), e.g. the cidr file in firewall rules.

Had the CIDR file in mind when I wrote this, but that doesn't compute as a factory to me. I'm actually having second thoughts on whether that should be a YaML at all vs a variable, but let's postpone/move this conversation.

Can you add those bits to the doc by editing this branch?

On it

Should we break this down into issues and start hammering our keyboards?
I'll have some time starting beginning of dec and keen to work on it.

Adding here a summary of what we discussed offline.

There are 2 angles for the discussion related to factories:

1. Factory interfaces, that is, are all the factories consistent on how they handle its inputs? This is relevant as if you're used to some of the existing factories it becomes natural to adopt a new factory. (do not break user expectation related to how our modules work).
2. Scope - What resources should each factory module manage? This is relevant to determine where the factories should live in the (mono) repo, as the current repo has some implicit definition of what lives where. (do not break user expectations related to how our Fabric repo works).

I'll focus only on the second since I don't have much to add to the interface discussion. For scope, we see that there are generally two types of factories:

• "Single resource" factories, which manage a partifcular low level component/product and its closely related resources (which are typically managed together). These are just helpers to manage a large number of resources which should be managed as "catle", for example, VPC firewall rules.
• "Composing" factories which tend to combine multiple different products in a way to solve a particular business problem or process following the "field proven" experience. This typically are related to "platform engineering", that is, a consistent process to onboarding a new workload runtime environment. For example, managing GCP projects, which are not just the project resource itself, but all the resources associated with a particular workload container, like its network subnets, essential contats, VPC SC perimeter membership etc, or a GKE namespace with all its governance rules.

Currently we have the first type of factories in the modules/ and blueprints/ folder, and the second type in blueprints/ and under FAST (fast/stages/). The fact that we have both under blueprints/ is expected since blueprints/ is the "nursery" of composing/modifying modules for new ideas which eventually get graduated into either modules/ or FAST.

So the plan would be that for factories that handle single resources, they should live directly under modules/. For factories that combine resources to implement a process, they should eventually live in FAST.

One of the challenges with having composing factories in FAST is that there is currently a (partly incorrect) assumption that FAST parts (stages) are always to be used to build an end-to-end landing zone, and not to be picked and used independently. One of the reasons for that is that not everything we call a FAST stage is actually a stage. While their folders have numbers suggesting a sequential implementation order, some of them are actually templates/blueprints that need to be customized to the customer to then be applied as one of the sequential stage.

Take project factory for example. A project factory is a FAST stage (which needs to be applied after network and security stages since it expects their outputs as its input when used as a workload "platform engineering" factory, but is also a blueprint that can be used on its own outside of a FAST landing zone as long as its expected inputs are provided via the variable surface (or even in other FAST places like the sandbox stage). In fact, the fast project factory stage is actually a sample implementation for "dev" of the blueprints/ hosted project factory model.

To make this distinction more clear (that these blueprints are a core part of FAST but can also be used on its own), we discussed moving these "template stages" under fast/blueprints/ (the implementation) and keep their actual invocations as a FAST end-to-end stage under fast/stages/. And to improve discoverability, also add references (via documentation) to the blueprints/ folder to these composing factories under fast/blueprints/).

Thanks for capturing the earlier discussion Roberto. My only nit is on the new folder naming: fast/stage-blueprints or fast/stage-templates (my preference) instead of fast/blueprints. This would avoid the confusion of "why do we have two blueprint folders" and clearly express the fact these are "stage 3 templates".

Thanks for capturing the earlier discussion Roberto. My only nit is on the new folder naming: fast/stage-blueprints or fast/stage-templates (my preference) instead of fast/blueprints. This would avoid the confusion of "why do we have two blueprint folders" and clearly express the fact these are "stage 3 templates".

Between fast/stage-blueprints and fast/stage-templates I prefer the first to make it more evident that it's something you can use it on its own without the whole FAST framework. For us (and FAST users) is clear what "stage 3" is but someone using it on its own would be "WTF is stage 3?" :D But in the end either are fine as long as we have the clarification/pointers under the blueprints/ folder. (If you arrive to fast/stage-templates via blueprints/README.md the actual fast folder name is a non-issue.)